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PROGRESS  REPORT:  6/30/96 

Overview  of  progress  on  ONR-funded  work  over  the  past  12  months 

The  goal  of  this  research  is  to  develop  strategies  for  the  control  of  dynamic  balance 
during  two-legged  and  four-legged  walking  based  on  a  hierarchy  of  simple  gait 
oscillators,  PID  controllers  and  neural  network  learning,  but  requiring  no  detailed 
dynamic  models.  A  common  learning  control  architecture  was  developed  for  the  UNH 
biped  and  quadruped  robots  which  emphasized  the  use  of  multiple  neural  network 
learning  modules  to  modulate  on-line  the  outputs  of  simple  gait  generators.  This 
feedforward  control  architecture  proved  to  be  successful  at  learning  nominal  dynamically 
balanced  walking  trajectories.  However,  it  proved  not  to  be  robust.  The  control 
architecture  was  then  modified  by  adding  control  elements  for  reacting  to  sensed 
disturbances  and/or  deviations  from  nominal  behavior,  patterned  after  results  in  the 
literature  from  human  walking  studies.  The  modified  walking  control  architecture  was 
refined  in  simulation,  while  an  improved  piezo-electric  "vestibular  system"  was 
developed  to  provide  better  perception  of  body  orientation.  The  modified  walking  control 
architecture  is  currently  being  implemented  on  the  experimental  biped  and  quadruped 
robots. 


Adaptive  Dynamic  Balance  of  Two  and  Four  Legged  Walking  Robots 
(ARPA/ONR  Grant  #N00014-95-l-1024) 

P.I.:  W.  Thomas  Miller,  III 
The  University  of  New  Hampshire 

Major  accomplishment  of  ONR-funded  work  over  the  past  12  months 

Two-legged  and  four-legged  walking  are  the  most  versatile  forms  of  land 
locomotion  in  the  sense  of  maneuverability  and  the  ability  to  traverse  irregular  terrain. 
Even  in  human  designed  spaces  with  horizontal  floors  and  regular  geometries,  legged 
locomotion  is  more  versatile  than  wheeled  locomotion.  Unfortunately,  the  problems  of 
practical  bipedal  and  quadrupedal  walking  with  dynamic  balance  have  so  far  eluded 
solution  using  classical  and  other  control  techniques.  Statically  unstable  but  dynamically 
stable  legged  locomotion  is  difficult  because  sensed  states  of  balance  or  imbalance  are 
related  in  complex  ways  to  the  recent  history  of  prior  control  actions.  In  addition,  once  a 
state  of  imbalance  is  sensed  reliably  it  is  often  too  late  to  recover.  Successful  walking 
primarily  requires  doing  the  right  thing  to  begin  with  (accurate  feedforward  control), 
while  using  closed  loop  reactive  control  mainly  for  minor  gait  adjustments,  and  using 
occasional  predetermined  recovery  motions  triggered  by  sensory  events  which  are 
predictive  of  subsequent  imbalance. 

In  general,  biped  walking  with  dynamic  balance  can  be  separated  into  three 
operating  regions  based  on  the  duration  of  the  single  support  phase  (one  foot  on  the 
ground)  during  each  step.  Similar  considerations  apply  to  quadruped  walking  if  more  than 
one  foot  is  lifted  simultaneously.  If  the  single  support  phase  is  short  relative  to  the  time 
constant  of  the  effective  “inverted  pendulum”  (the  true  dynamics  are  clearly  more 
complicated  than  those  of  a  simple  inverted  pendulum  due  to  the  distributed  mass  in  the 
multi-jointed  structure),  dynamic  balance  is  relatively  easy  in  that  the  biped  doesn’t  have 
time  to  fall  during  the  interval  in  which  one  foot  is  off  of  the  ground.  Single  support 
duration  is  easily  regulated  by  rapidly  retracting  and  then  extending  the  lifted  leg.  Balance 
is  mostly  a  function  of  where  the  lifted  foot  is  placed  over  sequences  of  steps  and  is  rather 
insensitive  to  the  details  of  the  biped  motion  while  a  foot  is  lifted.  This,  in  fact,  is  the 
problem  addressed  quite  successfully  by  Raibert’s  work  with  hopping  robots.  Such  a  gait, 
however,  is  generally  not  smooth  or  energy  efficient  (similar  to  a  person  constantly 
jogging,  independent  of  desired  speed  of  locomotion). 

If  the  single  support  duration  is  similar  in  magnitude  to  the  time  constant  of  the 
effective  “inverted  pendulum”  (on  the  order  of  0.4  seconds  for  our  biped)  dynamic 
balance  becomes  more  difficult.  After  a  foot  is  lifted  the  biped  “falls”  back  onto  the  lifted 
foot,  terminating  the  single  support  phase  in  accordance  with  the  time  constant  of  the 
system  and  the  initial  conditions  (structural  configuration  and  inertia)  before  the  foot  was 
lifted.  The  detailed  motions  of  the  biped  structure  thus  affect  the  elapsed  time  before  the 
biped  falls  back  onto  the  lifted  foot.  The  detailed  motions  of  the  biped  must  be  adapted  to 
match  the  effective  time  constant  of  the  structure  to  the  desired  single  support  duration. 


while  preserving  body  momentum  during  the  double  support  phase  in  order  to  ensure  a 
smooth  transition  from  one  step  to  the  next.  This  is  the  problem  which  has  received  the 
majority  of  our  attention  during  the  last  year,  with  good  results. 

Finally,  if  the  desired  single  support  duration  is  long  relative  to  the  effective  time 
constant  of  the  system,  the  biped  must  be  controlled  by  launching  into  and  maintaining  a 
state  of  quasi-static  balance  during  the  single  support  phase.  This  is  extremely  difficult 
since  achieving  a  reasonable  stride  length  requires  that  the  biped  undergo  a  substantial 
change  in  configuration  (and  redistribution  of  mass  and  momentum)  during  single 
support.  The  structure  must  be  balanced  precisely  at  every  control  cycle  during  this 
motion. 


We  have  been  investigating  the  use  of  neural  networks  for  on-line  gait  modulation 
during  walking,  using  a  10  degree-of-freedom  biped  robot  and  a  20  degree-of-freedom 
quadruped  robot  developed  in  our  laboratory.  For  sensing  balance,  these  robots  are 
equipped  with  foot  force  sensors  (4  per  foot)  and  piezo-electric  "vestibular  systems" 
which  produce  a  perceived  body  orientation  by  combining  information  from  linear 
accelerometers  and  angular  rate  gyroscopes.  The  approach  to  walking  control  that  we  are 
studying  is  to  use  simple  walking  gait  generators  based  on  approximate  kinematics  of  the 
structure  to  create  repetitive  walking  motions.  These  simple  walking  patterns  are  then 
modulated  by  multiple  neural  network  modules  trained  on-line,  utilizing  concepts  of  both 
supervised  and  reinforcement  learning,  to  avoid  states  of  sensed  imbalance.  Results  to 
date  indicate  that  our  approach  to  on-line  gait  adaptation  is  effective  at  automatically 
developing  the  detailed  nominal  motions  of  dynamically  balanced,  phase-locked, 
feedforward  gaits  for  two-legged  and  four-legged  robots  under  nominal  operating 
conditions,  requiring  very  little  a  priori  knowledge  of  the  robot  dynamics. 

Since  our  initial  control  architecture  depended  nearly  exclusively  on  learned 
feedforward  it  proved  not  to  be  robust,  limiting  overall  performance.  Deviations  from  the 
learned  nominal  behavior  would  inevitably  occur,  accumulating  to  the  point  of  failure  for 
more  challenging  gaits.  Most  recently,  the  primary  goal  of  our  research  has  been  to 
increase  the  robustness  of  the  walking  control  by  adding  control  elements  for  reacting  to 
sensed  disturbances  and/or  deviations  from  nominal  behavior.  Studies  reported  in  the 
literature  suggest  that  two  mechanisms  may  be  used  for  this  purpose  in  human  standing 
and  walking  (although  the  details  are  not  complete).  Small  disturbances  are  possibly 
handled  by  relatively  high  bandwidth  reactive  control  of  ankle  torque  based  on  vestibular 
and  proprioceptive  feedback.  Larger  disturbances  are  possibly  handled  by  lower 
bandwidth  changes  in  body  posture  involving  the  ankle,  knee  and  hip  joints.  These 
corrective  motions  appear  to  be  triggered  by  sensory  events  (again  from  the 
proprioceptive  and  vestibular  systems)  but  are  largely  feedforward  in  nature  subsequent  to 
the  initial  trigger. 

We  have  successfully  integrated  similar  control  strategies  into  the  controller  for 
our  biped  simulator  and  have  used  the  simulator  to  refine  details  which  are  poorly  defined 
in  the  available  literature  from  human  studies.  In  addition  we  have  developed  an 


improved  piezo-electric  "vestibular  system"  for  the  experimental  robots  providing  a 
continuous  perceived  body  orientation  using  a  combination  of  linear  accelerometers  and 
angular  rate  gyroscopes.  Finally,  we  are  upgrading  the  control  software  for  our 
experimental  biped  and  quadruped  walking  robots  in  order  to  study  experimentally  the 
robustness  properties  of  the  control  combining  learned  feedforward  gaits,  reactive  ankle 
torque  control  about  the  learned  nominal  gait  trajectories,  and  preprogrammed 
disturbance  rejection  (balance  recovery). 


The  UNH  biped  robot. 


Accelerometers  and  Rate  Gyroscopes 
On  Side  and  Rear 


\ 


UNH  experimental  biped  system. 


The  UNH  quadruped  robot. 
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experimental  quadruped  system. 


